Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tool] Add a package inclusion filter #4904

Merged
merged 1 commit into from
Sep 12, 2023

Conversation

stuartmorgan
Copy link
Contributor

Creates a package filter flag that is an opt-in mirror to the exclude flag, for use with the automatic package selection options used in CI. Like exclude, it allows for YAML files as input.

This allows for creating split test runs in CI (e.g., during incremental migrations), where the same file is an inclusion filter for one run and an exclusion filter for the other, guaranteeing that tests are in one or the other without the possibility of some tests falling through the cracks.

Creates a package filter flag that is an opt-in mirror to the `exclude`
flag, for use with the automatic package selection options used in CI.
Like `exclude`, it allows for YAML files as input.

This allows for creating split test runs in CI (e.g., during
incremental migrations), where the same file is an inclusion filter for
one run and an exclusion filter for the other, guaranteeing that tests
are in one or the other without the possibility of some tests falling
through the cracks.
@stuartmorgan
Copy link
Contributor Author

This is intended as a generic solution to the problem of #4820, where we want tests run in one of two modes, while still making it very hard to accidentally lose any tests (it would require eliminating one of the runs without removing the flag from the other, which I think we can effectively guard against with comments on the scripts).

So for that PR you could make, e.g., a still_requires_api_33_emulators.yaml, and then the shards with 33 would run a script that passes that with --filter-packages-to, while the 34 shards would instead pass the same file to --exclude.

@reidbaker
Copy link
Contributor

Test failure is clearly unrelated.

Potentially flakey "loadUrl with headers" test
https://firebase.corp.google.com/project/flutter-infra-staging/testlab/histories/bh.555af5d3de4d333b/matrices/7947959860904394775/details?stepId=bs.52b034256ab1b4b8&testCaseId=6&tabId=logs&query=et.f3a383dc00dd0737&deeplink=true

java.lang.Exception: ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following TestFailure was thrown running a test:
Expected: 'http://0.0.0.0:37207/headers'
Actual: 'http://0.0.0.0:37207/hello.txt'
Which: is different.
Expected: ... 0:37207/headers
Actual: ... 0:37207/hello.txt

@stuartmorgan stuartmorgan added the autosubmit Merge PR when tree becomes green via auto submit App label Sep 12, 2023
@auto-submit auto-submit bot merged commit e04ba88 into flutter:main Sep 12, 2023
70 checks passed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 12, 2023
@stuartmorgan stuartmorgan deleted the tool-package-filter-flag branch September 12, 2023 16:06
auto-submit bot pushed a commit to flutter/flutter that referenced this pull request Sep 12, 2023
flutter/packages@ef0c65e...e04ba88

2023-09-12 [email protected] [tool] Add a package inclusion filter (flutter/packages#4904)
2023-09-12 [email protected] [flutter_markdown] Fix changelog regarding minimum supported SDK version (flutter/packages#4851)
2023-09-12 [email protected] [ios_platform_images] Add integration tests (flutter/packages#4899)
2023-09-12 [email protected] [image_picker] Copy exif tags in categories II and III (flutter/packages#4738)
2023-09-11 [email protected] [google_maps_flutter_android] Fix for testToggleInfoWindow persistently flaky  (flutter/packages#4768)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-packages-flutter-autoroll
Please CC [email protected],[email protected] on the revert to ensure that a human
is aware of the problem.

To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
sybrands-place pushed a commit to sybrands-place/packages that referenced this pull request Sep 14, 2023
* main: (83 commits)
  go_router_builder: support the latest pkg:analyzer (flutter#4921)
  Replace collection type lints with more general lint (flutter#4912)
  Roll Flutter from 219efce to 4e7a07a (30 revisions) (flutter#4910)
  [camerax] Implement `startVideoCapturing` and `onVideoRecordedEvent` (flutter#4815)
  [tool] Add a package inclusion filter (flutter#4904)
  [flutter_markdown] Fix changelog regarding minimum supported SDK version (flutter#4851)
  [ios_platform_images] Add integration tests (flutter#4899)
  [image_picker] Copy exif tags in categories II and III (flutter#4738)
  [google_maps_flutter_android] Fix for testToggleInfoWindow persistently flaky  (flutter#4768)
  Roll Flutter from 7c28e8e to 219efce (16 revisions) (flutter#4901)
  [url_launcher] migrating iOS tests from objc to swift (flutter#4758)
  Roll Flutter from da676f7 to 7c28e8e (20 revisions) (flutter#4879)
  Bump actions/upload-artifact from 3.1.2 to 3.1.3 (flutter#4863)
  Roll Flutter from aea4552 to da676f7 (28 revisions) (flutter#4874)
  [webview_flutter_android] Added the functionality to fullscreen html5 video (flutter#3879)
  [tool] Add Android dependency (gradle) option to update dependencies command (flutter#4757)
  [camerax] Implement resolution configuration (flutter#3799)
  Manual roll Flutter from 685ce14 to aea4552 (64 revisions) (flutter#4870)
  [rfw, ci] Regenerate goldens, manually roll flutter#4835 (flutter#4862)
  Bump actions/checkout from 3.6.0 to 4.0.0 (flutter#4845)
  ...
@reidbaker reidbaker mentioned this pull request Sep 15, 2023
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
autosubmit Merge PR when tree becomes green via auto submit App
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants